A CCG-based Quality Estimation Metric for Statistical Machine Translation

We describe a metric for estimating the quality of Statistical Machine Translation (SMT) output based on syntactic features extracted using Combinatory Categorial Grammar (CCG). CCG has been demonstrated to be better suited to deal with SMT texts than context free phrase structure grammar formalisms. We use CCG features to estimate the grammaticality of the translations by dividing them into maximal grammatical chunks extracted from their CCG parse chart. We compare the performance of our CCG features with strong baseline and linguistic feature sets on French–English and Arabic–English data sets annotated with various quality scores. The results show that our CCG features outperform the baseline and linguistic features in most of the experiments. Furthermore, we demonstrate that our CCG features complement other types of features: combining CCG features with the baseline and other linguistic features furthers their performance.